TESTed—An educational testing framework with language-agnostic test suites for programming exercises

TSNLP - Test Suites for Natural Language Processing

The growing la.ngua<gc l.echnology indusl;ry n(!eds nl(!nslireili{~iil; tools 1;o allow rf!se;-i, rch(ws, (qigiI [ ( ~ ( W S 1 III~LII?I,~2~(~FS 1 a,nd CllSI;OIII(!I 'S i ;o l,l'~mk ,l,,v<~lOpl/ICllI;, (wa, llla, t e a , n d ;-tSSllre (llla,ii|;y, ;'~ll(I ;is;s(~ss suit;al)ilil;y for <t variety of ~tl)l)licalions. The 'I'SNIA' ('['(!s|. Suil;(!s for N;d;ural l,~uiguag{: ])l'O(:(~S<";illg~) 1)ro...

catnap: Generating Test Suites of Constrained Combinatorial Testing with Answer Set Programming

We develop an approach to test suite generation for Constrained Combinatorial Testing (CCT), one of the most widely studied combinatorial testing techniques, based on Answer Set Programming (ASP). The resulting catnap system accepts a CCT instance in fact format and combines it with a first-order encoding for generating test suites, which can subsequently be solved by any off-the-shelf ASP syst...

Mutatis Mutandis: Comparing DBMS Test Suites with Mutation Testing

Testing consumes significant human and machine resources, especially for large, complex systems such as database servers. While a variety of testing approaches have been proposed to improve the efficiency of the testing process, it is difficult to evaluate these approaches. Mutation testing has been proposed as a way to assess the adequacy of a test suite, assigning a score that can be used to ...

A Framework for Automatically Repairing GUI Test Suites

Testing is an effective means for assuring the quality of software. In programs with Graphical User Interfaces (GUIs), event sequences serve as test cases for executing system tests. To aid in the test generation process, researchers have developed methods that automatically derive graph models from GUIs, which can then be traversed to create sequences for testing. Recent advances using these g...

Test Suites for NLP